Statistics professors are excited that the United States Supreme Court issued an opinion that ostensibly addressed statistical significance. One such example of the excitement is an article, in press, by Joseph B. Kadane, Professor in the Department of Statistics, in Carnegie Mellon University, Pittsburgh, Pennsylvania. See Joseph B. Kadane, “Matrixx v. Siracusano: what do courts mean by ‘statistical significance’?” 11[x] Law, Probability and Risk 1 (2011).
Professor Kadane makes the sensible point that the allegations of adverse events did not admit of an analysis that would imply statistical significance or its absence. Id. at 5. See Schachtman, “The Matrixx – A Comedy of Errors” (April 6, 2011)”; David Kaye, ” Trapped in the Matrixx: The U.S. Supreme Court and the Need for Statistical Significance,” BNA Product Safety and Liability Reporter 1007 (Sept. 12, 2011). Unfortunately, the excitement has obscured Professor Kadane’s interpretation of the Court’s holding, and has led him astray in assessing the importance of the case.
In the opening paragraph of his paper, Professor Kadane quotes from the Supreme Court’s opinion that “the premise that statistical significance is the only reliable indication of causation … is flawed,” Matrixx Initiatives, Inc. v. Siracusano, ___ U.S. ___, 131 S.Ct. 1309 (2011). The quote is accurate, but Professor Kadane proceeds to claim that this quote represents the holding of the Court. Kadane, supra at 1. The Court held no such thing.
Matrixx was a security fraud class action suit, brought by investors who claimed that the company misled them when they spoke to the market about the strong growth prospects of the company’s product, Zicam cold remedy, when they had information that raised concerns that might affect the product’s economic viability and its FDA license. The only causation required for the plaintiffs to show was an economic loss caused by management’s intentional withholding of “material” information that should have been disclosed under all the facts and circumstances. Plaintiffs do not have to prove that the medication causes the harm alleged in personal injury actions. Indeed, it might turn out to be indisputable that the medication does not cause the alleged harm, but earlier, suggestive studies would provoke regulatory intervention and even a regulatory decision to withdraw the product from the market. Investors obviously could be hurt under this scenario as much as, if not more than, if the medication caused the harms alleged by personal-injury plaintiffs.
Kadane’s assessment goes awry in suggesting that the Supreme Court issued a holding about facts that were neither proven nor necessary for it to reach its decision. Court can, and do, comment, note, and opine about many unnecessary facts or allegations in reaching a holding, but these statements are obiter dicta, if they are not necessary to the disposition of the case. Because medical causation was not required for the Supreme Court to reach its decision, its presence or absence was not, and could not, be part of the Court’s holding.
Kadane makes a similar erroneous statement that the lower appellate courts, which earlier had addressed “statistical significance,” properly or improperly understood, found that “statistical significance in the strict sense [was] neither necessary … nor sufficient … to require action to remove a drug from the market.” Id. at 6. The earlier appellate decisions addressed securities fraud, however, not regulatory action of withdrawal of a product. Kadane’s statement mistakes what was at issue, and what was decided, in all the cases discussed.
Kadane seems at least implicitly to recognize that medical causation is not at issue when he states that “the FDA does not require proof of causation but rather reasonable evidence of an association before a warning is issued.” Id. at 7 (internal citation omitted). All that had to have happened for the investors to have been harmed by the Company’s misleading statements was for Matrixx Initiatives to boast about future sales, and to claim that there were no health issues that would lead to regulatory intervention, when they had information raising doubts about their claim of no health issues. See FDA Regulations, 21 U.S.C. § 355(d), (e)(requiring drug sponsor to show adequate testing, labeling, safety, and efficacy); see also 21 C.F.R. § 201.57(e) (requiring warnings in labeling “as there is reasonable evidence of an association of a serious hazard with a drug; a causal relationship need not have been proved.”); 21 C.F.R. § 803.3 (adverse event reports address events possibly related to the drug or the device); 21 C.F.R. § 803.16 (adverse event report is not an admission of causation).
Kadane’s analysis of the case goes further astray when he suggests that the facts were strong enough for the case to have survived summary judgment. Id. at 9. The Matrixx case was a decision on the adequacy of the pleadings, not of the adequacy of the facts proven. Elsewhere, Kadane acknowledges the difference between a challenge to the pleadings and the legal sufficiency of the facts, id. at 7 & n.8, but Kadane asserts, without explanation, that the difference is “technical” and does not matter.” Not true. The motion to dismiss is made upon receipt of the plaintiffs’ complaint, but the motion for summary judgment is typically made at the close of discovery, on the eve of trial. The allegations can be conclusory, and they need have only plausible support in other alleged facts to survive a motion to dismiss. The case, however, must have evidence of all material facts, as well as expert witness opinion that survives judicial scrutiny for scientific validity under Rule 702, to survive a motion for summary judgment, which comes much later in the natural course of any litigated case.
Kadane appears to try to support the conflation of dismissals on the pleadings and summary judgments by offering a definition of summary judgment that is not quite accurate, and potentially misleading: “The idea behind summary judgment is that, even if every fact alleged by the opposing party were found to be true, the case would still fail for legal reasons.” Id. at 2. The problem is that at the summary judgment stage, as opposed to the pleading stage, the party with the burden of proof cannot rest upon his allegations, but must come forward with facts, not allegations, to support every essential element of his case. A plaintiff in a personal injury action (not a securities fraud case), for example, may easily survive a motion to dismiss by alleging medical causal connection, but at the summary judgment stage, that plaintiff must serve a report of an appropriately qualified expert witness, who in turn has presented a supporting opinion, reliably ground in science, to survive both evidentiary challenges and a dispositive motion.
Kadane concludes that the Matrixx decision’s “fact-based consideration” is consistent with a “Bayesian decision-theoretic approach that models how to make rational decisions under uncertainty.” Id. at 9. I am 99.99999% certain that Justice Sotomayor would not have a clue about what Professor Kadane was saying. Although statistical significance may have played no role in the Court’s holding, and in Kadane’s Bayesian decision-theoretic approach, I am 100% certain that the irrelevance of statistical significance to the Court’s and Prof. Kadane’s approaches is purely coincidental.
